Application Of Deep Neural Networks Research Articles

Machine Learning Approaches for ADHD Detection

This report explores the application of deep learning convolutional neural networks (CNNs) for analyzing brain MRI scans to assist in the diagnosis of Attention Deficit Hyperactivity Disorder (ADHD) syndrome. ADHD is a neuro developmental disorder that affects millions of individuals worldwide, characterized by difficulties in attention, hyperactivity, and impulsivity. Traditional methods of diagnosis rely on subjective assessments and behavioral observations, leading to inaccuracies and delays in treatment. Leveraging the power of deep learning CNNs for MRI analysis offers the potential for more objective and efficient diagnosis, facilitating timely intervention and personalized treatment strategies

Improving Adversarial Robustness of ECG Classification Based on Lipschitz Constraints and Channel Activation Suppression.

Deep neural networks (DNNs) are increasingly important in the medical diagnosis of electrocardiogram (ECG) signals. However, research has shown that DNNs are highly vulnerable to adversarial examples, which can be created by carefully crafted perturbations. This vulnerability can lead to potential medical accidents. This poses new challenges for the application of DNNs in the medical diagnosis of ECG signals. This paper proposes a novel network Channel Activation Suppression with Lipschitz Constraints Net (CASLCNet), which employs the Channel-wise Activation Suppressing (CAS) strategy to dynamically adjust the contribution of different channels to the class prediction and uses the 1-Lipschitz's ℓ∞ distance network as a robust classifier to reduce the impact of adversarial perturbations on the model itself in order to increase the adversarial robustness of the model. The experimental results demonstrate that CASLCNet achieves ACCrobust scores of 91.03% and 83.01% when subjected to PGD attacks on the MIT-BIH and CPSC2018 datasets, respectively, which proves that the proposed method in this paper enhances the model's adversarial robustness while maintaining a high accuracy rate.

Application of Deep Quantum Neural Networks to Finance

The recent development of quantum computing gives us an opportunity to explore its potential applications to many fields, with the field of finance being no exception. In this paper, we apply the deep quantum neural network proposed by Beer et al. (2020) and discuss such potential in the context of simple experiments, such as learning implied volatilities and option prices. Furthermore, Greeks, such as delta and gamma, which are important measures in risk management, can be computed analytically with the neural network, and our numerical experiments show that the deep quantum neural network is a promising technique for solving such numerical problems arising in finance.

A Classification method for Insects using Data Augmentation and Deep Neural Networks

In the nature, there exists a huge number of species of insects. Insects have caused damage for human and crops. Traditional identification methods of insects require expert knowledge and time consuming. Therefore, the automatic identification and classification have been more and more necessary. In recent year, one of the efficient approaches to classify insects is the application of Deep neural networks. The paper presents the improvements for the classification of insect images. Firstly, we apply the data augmentation to improve the number of insect images. Then, we applied various deep neural networks to improve the classification accuracy of insect classification. Obtained classification accuracy of 98% on public insect datasets shows the efficiency of the proposed method.

VerSA: Versatile Systolic Array Architecture for Sparse and Dense Matrix Multiplications

A key part of modern deep neural network (DNN) applications is matrix multiplication. As DNN applications are becoming more diverse, there is a need for both dense and sparse matrix multiplications to be accelerated by hardware. However, most hardware accelerators are designed to accelerate either dense or sparse matrix multiplication. In this paper, we propose VerSA, a versatile systolic array architecture for both dense and sparse matrix multiplications. VerSA employs intermediate paths and SRAM buffers between the rows of the systolic array (SA), thereby enabling an early termination in sparse matrix multiplication with a negligible performance overhead when running dense matrix multiplication. When running sparse matrix multiplication, 256 × 256 VerSA brings performance (i.e., an inverse of execution time) improvement and energy saving by 1.21×–1.60× and 7.5–30.2%, respectively, when compared to the conventional SA. When running dense matrix multiplication, VerSA results in only a 0.52% performance overhead compared to the conventional SA.

Evolution of laughter from play

ABSTRACT In this hypothesis, I discuss how laughter from physical play could have evolved to being induced via visual or even verbal stimuli, and serves as a signal to highlight incongruity that could potentially pose a threat to survival. I suggest how laughter’s induction could have negated the need for physical contact in play, evolving from its use in tickling, to tickle-misses, and to taunting, and I discuss how the application of deep learning neural networks trained on images of spectra of a variety of laughter types from a variety of individuals or even species, could be used to determine such evolutionary pathways via the use of latent space exploration.

Generating Synthetic Sperm Whale Voice Data Using StyleGAN2-ADA

The application of deep learning neural networks enables the processing of extensive volumes of data and often requires dense datasets. In certain domains, researchers encounter challenges related to the scarcity of training data, particularly in marine biology. In addition, many sounds produced by sea mammals are of interest in technical applications, e.g., underwater communication or sonar construction. Thus, generating synthetic biological sounds is an important task for understanding and studying the behavior of various animal species, especially large sea mammals, which demonstrate complex social behavior and can use hydrolocation to navigate underwater. This study is devoted to generating sperm whale vocalizations using a limited sperm whale click dataset. Our approach utilizes an augmentation technique predicated on the transformation of audio sample spectrograms, followed by the employment of the generative adversarial network StyleGAN2-ADA to generate new audio data. The results show that using the chosen augmentation method, namely mixing along the time axis, makes it possible to create fairly similar clicks of sperm whales with a maximum deviation of 2%. The generation of new clicks was reproduced on datasets using selected augmentation approaches with two neural networks: StyleGAN2-ADA and WaveGan. StyleGAN2-ADA, trained on an augmented dataset using the axis mixing approach, showed better results compared to WaveGAN.

STCS-Net: a medical image segmentation network that fully utilizes multi-scale information

In recent years, significant progress has been made in the field of medical image segmentation through the application of deep learning and neural networks. Numerous studies have focused on optimizing encoders to extract more comprehensive key information. However, the importance of decoders in directly influencing the final output of images cannot be overstated. The ability of decoders to effectively leverage diverse information and further refine crucial details is of paramount importance. This paper proposes a medical image segmentation architecture named STCS-Net. The designed decoder in STCS-Net facilitates multi-scale filtering and correction of information from the encoder, thereby enhancing the accuracy of extracting vital features. Additionally, an information enhancement module is introduced in skip connections to highlight essential features and improve the inter-layer information interaction capabilities. Comprehensive evaluations on the ISIC2016, ISIC2018, and Lung datasets validate the superiority of STCS-Net across different scenarios. Experimental results demonstrate the outstanding performance of STCS-Net on all three datasets. Comparative experiments highlight the advantages of our proposed network in terms of accuracy and parameter efficiency. Ablation studies confirm the effectiveness of the introduced decoder and skip connection module. This research introduces a novel approach to the field of medical image segmentation, providing new perspectives and solutions for future developments in medical image processing and analysis.

Physics-constrained and flow-field-message-informed graph neural network for solving unsteady compressible flows

With the increasing use of deep neural networks as surrogate models for accelerating computational simulations in mechanics, the application of artificial intelligence in computational fluid dynamics has seen renewed interest in recent years. However, the application of deep neural networks for flow simulations has mainly concentrated on relatively simple cases of incompressible flows. The strongly discontinuous structures that appear in compressible flows dominated by convection, such as shock waves, introduce significant challenges when approximating the nonlinear solutions or governing equations. In this work, we propose a novel physics-constrained, flow-field-message-informed (FFMI) graph neural network for spatiotemporal flow simulations of compressible flows involving strong discontinuities. To enhance the nonlinear approximation capability of strong discontinuities, a shock detector method is leveraged to extract the local flow-field messages. These messages are embedded into the graph representation to resolve the discontinuous solutions accurately. A new FFMI sample-and-aggregate-based message-passing layer, which aggregates the edge-weighted attributes with node features on different hop layers, is then developed to diffuse and process the flow-field messages. Furthermore, an end-to-end paradigm is established within the encoder–decoder framework to transform the extracted information from the flow field into latent knowledge about the underlying fluid mechanics. Finally, a variety of one- and two-dimensional cases involving strong shock waves are considered to demonstrate the effectiveness and generalizability of the proposed FFMI graph neural network.

Lightweight blueprint residual network for single image super-resolution

The application of deep convolutional neural networks (CNNs) makes the lightweight single image super-resolution (SISR) task develop rapidly in recent years. However, existing lightweight SISR networks for the terminals with limited computing resources are still not efficient enough. In order to further alleviate the model complexity while keeping remarkable reconstruction performance, in this paper, we propose a lightweight blueprint residual network (LBRN), which is a powerful CNN-based SR model while simultaneously maintaining extremely lightweight. Specifically, we design a novel lightweight blueprint residual block (LBRB) to learn the high-frequency information efficiently for lightweight SISR. The LBRB with fewer parameters consists of more efficient convolution operations to extract high-frequency information which helps recover more visual details. In addition, we propose an effective feature fusion residual block (EFFRB) which is composed of three LBRBs with convolutional kernels of different sizes. The EFFRB, which is designed with effective multi-scale receptive fields, can availably extract and fuse multi-scale high-frequency information to obtain discriminative feature, and it further improves model performance. Extensive experiments show that our LBRN achieves superior reconstruction performance quantitatively and visually compared with other lightweight state-of-the-art SR methods.

Application of deep neural networks for automatic irony detection in Russian texts

The paper examines automatic methods for classifying Russian-language sentences into two classes: ironic and non-ironic. The discussed methods can be divided into three categories: classifiers based on language model embeddings, classifiers using sentiment information, and classifiers with embeddings trained to detect irony. The components of classifiers are neural networks such as BERT, RoBERTa, BiLSTM, CNN, as well as an attention mechanism and fully connected layers. The irony detection experiments were carried out using two corpora of Russian sentences: the first corpus is composed of journalistic texts from the OpenCorpora open corpus, the second corpus is an extension of the first one and is supplemented with ironic sentences from the Wiktionary resource. The best results were demonstrated by a group of classifiers based on embeddings of language models with the maximum F-measure of 0.84, achieved by a combination of RoBERTa, BiLSTM, an attention mechanism and a pair of fully connected layers in experiments on the extended corpus. In general, using the extended corpus produced results that were 2–5% higher than those of the basic corpus. The achieved results are the best for the problem under consideration in the case of the Russian language and are comparable to the best one for English.

Traffic Sign Recognition and Classification using Deep Neural Networks

Deep Neural Networks such as Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), have been applied in various applications, including machine vision and computer vision. One of the most notable emerging applications of deep neural networks is Advanced Driver Assistance System (ADAS). In ADAS model, the deep neural networks have outperformed the manual human performance in terms of traffic sign recognition and classification. to the problem of traffic sign recognition, with promising results. This study has proposed a novel Convolutional Neural Network (CNN) architecture for recognizing and classifying traffic signs. The German Traffic Sign Recognition Benchmark dataset is used for experimental analysis. The outcome shows that the proposed model performs comparatively better. The results also indicate that the implemented CNN model with Adam optimizer and RankMax activation performs well for recognizing and classifying traffic sign images.

A Physics-Informed Low-Shot Adversarial Learning for sEMG-Based Estimation of Muscle Force and Joint Kinematics.

Muscle force and joint kinematics estimation from surface electromyography (sEMG) are essential for real-time biomechanical analysis of the dynamic interplay among neural muscle stimulation, muscle dynamics, and kinetics. Recent advances in deep neural networks (DNNs) have shown the potential to improve biomechanical analysis in a fully automated and reproducible manner. However, the small sample nature and physical interpretability of biomechanical analysis limit the applications of DNNs. This paper presents a novel physics-informed low-shot adversarial learning method for sEMG-based estimation of muscle force and joint kinematics. This method seamlessly integrates Lagrange's equation of motion and inverse dynamic muscle model into the generative adversarial network (GAN) framework for structured feature decoding and extrapolated estimation from the small sample data. Specifically, Lagrange's equation of motion is introduced into the generative model to restrain the structured decoding of the high-level features following the laws of physics. A physics-informed policy gradient is designed to improve the adversarial learning efficiency by rewarding the consistent physical representation of the extrapolated estimations and the physical references. Experimental validations are conducted on two scenarios (i.e. the walking trials and wrist motion trials). Results indicate that the estimations of the muscle forces and joint kinematics are unbiased compared to the physics-based inverse dynamics, which outperforms the selected benchmark methods, including physics-informed convolution neural network (PI-CNN), vallina generative adversarial network (GAN), and multi-layer extreme learning machine (ML-ELM).

An Innovative Inducer of Platelet Production, Isochlorogenic Acid A, Is Uncovered through the Application of Deep Neural Networks.

(1) Background: Radiation-induced thrombocytopenia (RIT) often occurs in cancer patients undergoing radiation therapy, which can result in morbidity and even death. However, a notable deficiency exists in the availability of specific drugs designed for the treatment of RIT. (2) Methods: In our pursuit of new drugs for RIT treatment, we employed three deep learning (DL) algorithms: convolutional neural network (CNN), deep neural network (DNN), and a hybrid neural network that combines the computational characteristics of the two. These algorithms construct computational models that can screen compounds for drug activity by utilizing the distinct physicochemical properties of the molecules. The best model underwent testing using a set of 10 drugs endorsed by the US Food and Drug Administration (FDA) specifically for the treatment of thrombocytopenia. (3) Results: The Hybrid CNN+DNN (HCD) model demonstrated the most effective predictive performance on the test dataset, achieving an accuracy of 98.3% and a precision of 97.0%. Both metrics surpassed the performance of the other models, and the model predicted that seven FDA drugs would exhibit activity. Isochlorogenic acid A, identified through screening the Chinese Pharmacopoeia Natural Product Library, was subsequently subjected to experimental verification. The results indicated a substantial enhancement in the differentiation and maturation of megakaryocytes (MKs), along with a notable increase in platelet production. (4) Conclusions: This underscores the potential therapeutic efficacy of isochlorogenic acid A in addressing RIT.

SecureNet: Proactive intellectual property protection and model security defense for DNNs based on backdoor learning

With the widespread application of deep neural networks (DNNs), the risk of privacy breaches against DNN models is constantly on the rise, resulting in an increasing need for intellectual property (IP) protection for such models. Although neural network watermarking techniques are widely used to safeguard the IP of DNNs, they can only achieve passive protection and cannot actively prevent unauthorized users from illicit use or embezzlement of the trained DNN models. Therefore, the development of proactive protection techniques to prevent IP infringement is imperative. To this end, we propose SecureNet, a key-based access license framework for DNN models. The proposed approach involves injecting license keys into the model through backdoor learning, enabling correct model functionality only when the appropriate license key is included in the input. To ensure the reusability of DNN models, we also propose a license key replacement algorithm. In addition, based on SecureNet, we designed defense mechanisms against adversarial attacks and backdoor attacks, respectively. Furthermore, we introduce a fine-grained authorization method that enables flexible granting of model permissions to different users. We have designed four license-key schemes with different privileges, tailored to various scenarios. We evaluated SecureNet on five benchmark datasets including MNIST, Cifar10, Cifar100, FaceScrub, and CelebA, and assessed its performance on six classic DNN models: LeNet-5, VGG16, ResNet18, ResNet101, NFNet-F5, and MobileNetV3. The results demonstrate that our approach outperforms the state-of-the-art model parameter encryption methods by at least 95% in terms of computational efficiency. Additionally, it provides effective defense against adversarial attacks and backdoor attacks without compromising the model’s overall performance.

Functions as a service for distributed deep neural network inference over the cloud‐to‐things continuum

AbstractThe use of serverless computing has been gaining popularity in recent years as an alternative to traditional Cloud computing. We explore the usability and potential development benefits of three popular open‐source serverless platforms in the context of IoT: OpenFaaS, Fission, and OpenWhisk. To address this we discuss our experience developing a serverless and low‐latency Distributed Deep Neural Network (DDNN) application. Our findings indicate that these serverless platforms require significant resources to operate and are not ideal for constrained devices. In addition, we archived a 55% improvement compared to Kafka‐ML's performance under load, a framework without dynamic scaling support, demonstrating the potential of serverless computing for low‐latency applications.

Latency-Constrained Neural Architecture Search Method for Efficient Model Deployment on RISC-V Devices

The rapid development of the RISC-V instruction set architecture (ISA) has garnered significant attention in the realm of deep neural network applications. While hardware-aware neural architecture search (NAS) methods for ARM, X86, and GPUs have been extensively explored, research specifically targeting RISC-V remains limited. In light of this, we propose a latency-constrained NAS (LC-NAS) method specifically designed for RISC-V. This method enables efficient network searches without the requirement of network training. Concretely, in the training-free NAS framework, we introduce an RISC-V latency evaluation module that includes two implementations: a lookup table and a latency predictor based on a deep neural network. To obtain real latency data, we have designed a specialized data collection pipeline for RISC-V devices, which allows for precise end-to-end hardware latency measurements. We validate the effectiveness of our method in the NAS-Bench-201 search space. Experimental results demonstrate that our method can efficiently search for latency-constrained networks for RISC-V devices within seconds while maintaining high accuracy. Additionally, our method can easily integrate with existing training-free NAS approaches.

Application of deep neural network to discriminating chalcopyrite deposits, ore types, and textures

The trace elements of chalcopyrite have indicative significance in the formation process and genetic types of its deposits. They can also function as indicator minerals for mineral exploration. This study investigates the chalcopyrite in the global representative Ni-Cu sulfide deposits and Reef-type PGE deposits, and establishes deep neural network (DNN) models to discriminate the chalcopyrite deposits, ore types, and textures. The feature importance of DNN models is analyzed by calculating the mean SHapley Additive ExPlanations (SHAP) value of 26 elements in chalcopyrite, revealing the important discriminant elements in DNN models. Dimensionality reduction algorithms such as principal component analysis (PCA) are employed to the dataset and determine the decision boundary for the subsequent visualization analysis. A series of comprehensive experiments are conducted and the results reveal the DNN models to discriminate chalcopyrite deposits and ore types at accuracies of 99 % and 98 %, respectively. Based on the sulfide texture classification scheme proposed by the previous studies, we classify the chalcopyrite textures using the DNN models, with an accuracy of 96 %. The high-precision classification of sulfide textures helps to systematically explore the formation mechanism of sulfide-rich magmatic ores. Moreover, we compare our DNN models with traditional machine learning algorithms using the chalcopyrite dataset. This further illustrates the successful application of DNN models in the discrimination of chalcopyrite deposits, ore types, and textures.

MA‐CAT: Misclassification‐Aware Contrastive Adversarial Training

Vulnerability to adversarial examples poses a significant challenge to the secure application of deep neural networks. Adversarial training and its variants have shown great potential in addressing this problem. However, such approaches, which directly optimize the decision boundary, often result in overly complex adversarial decision boundaries that are detrimental to generalization. To deal with this issue, a novel plug‐and‐play method known as Misclassification‐Aware Contrastive Adversarial Training (MA‐CAT) from the perspective of data distribution optimization is proposed. MA‐CAT leverages supervised decoupled contrastive learning to cluster nature examples within the same class in the logit space, indirectly increasing the margins of examples. Moreover, by taking into account the varying difficulty levels of adversarial training for different examples, MA‐CAT adaptively customizes the strength of adversarial training for each example using an instance‐wise misclassification‐aware adaptive temperature coefficient. Extensive experiments on the CIFAR‐10, CIFAR‐100, and SVHN datasets demonstrate that MA‐CAT can be easily integrated into existing models and significantly improves robustness with minimal computational cost.

Applicability of deep neural networks for lithofacies classification from conventional well logs: An integrated approach

Parametric understanding for specifying formation characteristics can be perceived through conventional approaches. Significantly, attributes of reservoir lithology are practiced for hydrocarbon exploration. Well logging is conventional approach which is applicable to predict lithology efficiently as compared to geophysical modeling and petrophysical analysis due to cost effectiveness and suitable interpretation time. However, manual interpretation of lithology identification through well logging data requires domain expertise with an extended length of time for measurement. Therefore, in this study, Deep Neural Network (DNN) has been deployed to automate the lithology identification process from well logging data which would provide support by increasing time-effective for monitoring lithology. DNN model has been developed for predicting formation lithology leading to the optimization of the model through the thorough evaluation of the best parameters and hyperparameters including the number of neurons, number of layers, optimizer, learning rate, dropout values, and activation functions. Accuracy of the model is examined by utilizing different evaluation metrics through the division of the dataset into the subdomains of training, validation and testing. Additionally, an attempt is contributed to remove interception for formation lithology prediction while addressing the imbalanced nature of the associated dataset as well in the training process using class weight. It is assessed that accuracy is not a true and only reliable metric to evaluate the lithology classification model. The model with class weight recognizes all the classes but has low accuracy as well as a low F1-score while LSTM based model has high accuracy as well as a high F1-score.